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Abstract 

This paper reports the results of a study of changes in the social distribution of mathematics achievement 
for a cohort of public h}gh school students. Using Hierarchical Linear Modeling (HLM) the study sought 
to identify school characteristics which were correlated with changes in achievement differences between 
white and black studes^ts; students from advantaged and disadvantaged socioeconomic backgrounds 
(SES); and male and female students; as they progressed from grade 9 to grade 11. The school 
characteristics considered included indicators of school/coir nunity context, school normative climate 
("effective schools" indicators), and school instructional setting. While many of the indicators were 
found to be related to achievement discrepancies at a given point in time, few yielded significant results 
for changes in the distribution of achievexnent. 

School location (suburban) , the stability of the student population, and the percent of low SES children 
in the school, were significant predictors of change over time. The results for one effective schools 
indictor (student perception of principal and teacher expectations), while not clear^ut, seemed to be 
predictive of movement toward a more equitable distribution as regards student SES. It is suggested that 
effective schools indicators which are more focussed toward equity issues, and those based on student 
(rather than teacher) input, may be needed if predictors of change are to be found. 
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A Longitudinal Study of the Social Distribution of Mathematics 

Achievement for a Coho rt of Public High School Students 
Introduction 

The last two decades have witnessed the emergence of a body of research with the expressed purpose 
of identifying and estimating the impact of school characteristics which influence the distribution of 
student achievement within schools. Chief among the attributes considered are school policies and 
practices, school normative climate, and school and community context. Researchers have sought to link 
these factors to the "social distribution" of achievement within schools— frequently defined in terms of 
historic achievement disparities between minority and majority children; males and females in certain 
subject areas; and children from low, middle, and upper socioeconomic backgrounds (e.g., see 
Brookover et al., 1978; Coleman, Hoffer & Kilgore, 1982; Edmonds, 1979a, 1979b; Purkey & Smith, 
1983). 

While this literature has done much to promote understanding of the dynamics of schooling, the 
majority of these studies have relied upon data collected at one or two points in time. It is generally 
recognized that designs of this type are not well suited to addressing questions of school ejects (Bryk & 
Raudenbush, 1987). Studies designed around cross-sectional variations in achievement outcomes 
encounter difficulties in disentagling school effects from the effects of student selection (e.g., Lee & 
Bryk, 1989). Alternatively, while pre- and post-test designs are better suited for addressing questions of 
simple change or difference, they are not well suited for studying the long-term growth which occurs in 
schools as students progress from grade to grade (see Bryk & Raudenbush, 1988). To do so requires 
investigations based on multiple data points and which focus on the trend or rate of change over several 
years. This position is adopted in the present paper. That is, it is argued that the question of school 
effects as regards the social distribution of achievement is not whether, say» the correlation of social class 
with achievement is weaker at one school than another at some specific period, but rather, whether as 
students progress through school the association grows stronger or weaker. From this perspective the 
school characteristics of interest would be those which correlate with the trend of this relationship. This 
becomes significant given that school effectiveness indicators generated by the current literature have 
been incorporated in school improvement programs across the country. The growth perspective adopted 
in this study would suggest that while these indicators may be correlated with the social distribudon of 
achievement^ they may or may not be related to changes in this distribution. 

In this paper we present the results of a longitudinal study of a cohort of public high school students 
followed over a four year period. The objective of the study was to identify and assess the impact of 
school characteristics which influence the trend of achievement differences in mathematics for blacks and 
whites; students from low and upper socioeconomic backgrounds, and males and females;. 

Background 

School Effects and the Social Distribution of Achievement 
For many years educators have debated the merits of public schools as regards the experiences of 
minorities, females, and students iirom less advantaged backgrounds. Quite often schools have been 
depicted as inept organizations which exert little influence on student achievement independent of fomily 
background and commimity context (Jencks, et al., 1972). Alternatively, they are often described as 
white, middle class, male dominated institutions which perpetuate the status quo by pronK>ting failure for 
minorities, women and students from the lower social classes (e.g., Bowles, 1977). In recent years, 
however, a number of studies have concluded that schools can, and often do, have a positive impact on 
student achievement (Coleman, Hoffer, & Kilgore, 1982). Focusing on achievement disparities between 
students from upper and lower socioeconomic backgrounds, investigators have reported that 
organizational and climatic aspects of the school setting tend to be correlated with an overall increase in 
student achievement and, in particular, with an increase in the equity of studoit achievement across the 
socioeconomic spectrum (see also Edmonds, 1981). Students from disadvantaged backgrounds, it has 
been reported, tend to benefit significantly from settings where academic expectations are high, there is a 
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general perception of the school enviroment as safe, and academically oriented courses are required 
(Hoffer, Greely, & Coleman, 1985). 

Although a number of independent research thriists have contributed to this literature (e.g., 
Brookover et al., 1978), the analyses of the High School and Beyond (HSB) surveys reported by 
Coleman et al. (1982) have been among the oiost widely debated. In a highly controversal report, 
Coleman et al. (1982) concluded that private schools. Catholic schools in particular, more closely 
resemble the notion of the "common school" than do public schools in that achievement tends to be more 
evenly dispersed across the social spectrum. These sector (public vs. private) differences, they noted, 
were on the order of about one grade level for the students studied. Using regression analysis with 
controls for 17 background variables, Coleman et al. (1982) found that sector differences in achievement 
decreased somewhat due to a "selection" effect but that the private advantage did not disappear. 

Coleman et al. (1982) investigated a number of possible explanations of the observed achievement 
differences between the public and private sectors. Through regression and other analyses the authors 
concluded that sector differences could be largely attributed to the climate and greater academic course 
requirements in the private sector. In particular, students in the private sector were found to have a 
greater sense of security at school, and to have taken more academically oriented courses. 

Criticisms of the Colenum et al. (1982) HSB analysis have been extensive and point to two primary 
weaknesses of this and other research directed at measuring school effects or identifying factors related to 
school effectiveness. First, school phenomena are hierarchical in nature; that is, students are nested 
within classrooms, classrooms are nested within schools and schools are subsumed under district policy. 
In the past, educators have lacked the statistical tools required to explicitly address the multilevel 
stnicture of schooling. As a consequence, many of the results regarding school influences on student 
learning have been questioned (Cronbach, 1976). However, recent advances in statistical modeling, in 
particular Hierarchical Linear Modelling (HLM), have provided researchers with the statistical tools 
needed to formulate and test more realistic models of schooling phenomena (Raudenbush & Bryk, 1986). 
This is possible because HLM allows the investigator of school effects to specify a structural model 
relating student characteristics to student level outcomes. The parameters from this model then become 
the outcome variables for models based on school characteristics. The model is very general in that in 
addition to studying average levels of student outcome achievement, complex multivariate relationships 
as expressed in regression slopes can also be modeled. For a complete review see Raudenbush and Bryk 
(1989). 

Several investigators have applied the HLM methodology to the HSB data studied by Coleman et al. 
(1982). Raudmbush and Bryk (1986), for example, reported that after controlling for school social class 
level, the Catholic school effect on average math achievement disappeared. However, the Catholic school 
impact on the social class/achievement regression slope held after considering school social class level 
and the number of honoework assignments students received. Employing this same methodology Lee and 
Bryk (1989) reported that achievement differences between minority and majority students were smaller 
when the school environment was perceived as orderly. They also reported that the association of social 
class and mathematics achievement was less if schools were small, noost students took many math 
courses, and the displinary climate was positive. 
The Study of Change 

The second primary criticism directed at school effectiveness and school effects studies concerns the 
issue of effecta over time. With regard to the relationship of social class to achievement, Lec (1986) has 
noted that while there is some disagreement about the magnitude of the effect, almost all observers 
concur that as students progress through school the relationship grows stronger, not weaker. Similarly, 
research on sex differences in mathematics achievement has shown that while females tend to perform as 
well as males in the early grades, they tend to fall progressively behind as they progress to the upper 
grades (sec Willms & Kerr, 1987). Unfortunately, the majority of the school effects and effectiveness 
studies have only considered cross-sectional variation in achievement, and when longitudinal designs 
have been used, typically, only two data points (e.g., pretest and postest) have been studied. Despite the 
information provided by these studies, as Bryk and Raudenbush (1987) note, they are not well suited to 
the study of trends or time related changes in school effects. Those based on cross-sectional designs are 
open to a number of criticisms. In particular, it can be argued that observed differences in school effects 
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ate a product of differences in student inputs and not other school related variables. The pretest/postest 
longitudinal approach, on the other hand, while an improvement over the cross-sectional design, is not 
well suited for addressing questions about rate of growth, as distinct from questions about simple change 
(see Bryk & Weisberg, 1977). To adequately address questions about rate of growth or change requires 
the simultaneous consideration of several dau points. 

To deal with these issues, consideration is given to an application of HLM to the problem of growth 
curve analysis. Arguing that studies of individual change have been plagiied with methodological, 
conceptual, and design problems, Bryk and Raudenbush (1987) propose a two stage formulation based on 
HLM. At stage 1 the individual is observed or measured on T different occasions. These observations are 
considered a function of an individual growth trajectory plus random error and is referred to as the 
within-subject model of change. It is assumed the growth rates and the associated parameters will vary 
among subjects. In the second sUge this variation is modeled as a function of differences between 
subjects in various attributes, i.e., the between<-subject model. According to Bryk and Raudenbush this 
development allows for "...examining the reliability of instruments for measuring status and change, 
investigating correlates of status and change, and testing hypotheses about the effects of backgroimd 
variables ^d experimental interventions on individual growth (p. 148).* 

In this paper a modified version of this noodel is employed. Instead of individual subjects, schools are 
the unit of analysis. Further, the growth criterion considered was not absolute achievement level, but the 
distribution of achievement for a cohort of public high school students as they progress from grade to 
grade. The distribution of achievement is operationally defined in terms of the following contrasts: 
whites vs. blacks; upper socio-economic background versus lower socio-economic background (hereafter 
high/low SES); and males versus females. In each insiance, the analysis sought to identify school 
characteristics which were associated with variation in the trend of a given contrast. 

Methodology 

Sample/Population 

The population consist of all public high schools in South Carolina which housed grades 9, 10, and 
1 1 during the period 1988 to 1990. Within each high school only students who were consistently 
promoted during the period of the study were included in the analyses. The sample of schools and 
students actually present for these analyses were determined by data availability. 
Achievement Data 

Student achievement data for the present study were obtained from two testing programs utilized in 
South Carolina during the study period. The Basic Skills Assessment Program (BSAP) consists of 
criterion-referenced tests administered annually statewide in the spring in grades 1, 2, 3, 6, 8 and a high 
school exit examination administered in grade 10. The Statewide Testing Program (STP) involves the 
administration of a norm-referenced test— The Comprehensive Test of Basic Skills (CTBS) during the 
study— administered annually statewide in the spring in grades 4, 5, 7, 9 and 11 . For the current study, 
interest is in the mathematics achievement, as reflected in performance on these instruments, of the 
cohort which entered a public high school in the fall of 1987. The tests, grades, and years of interest are 
as follows: BSAP Grade 8, Spring 1987; CTBS Grade 9, Spring 1988; EXIT Grade 10, Spring 1989; and 
CTBS Grade 11, Spring 1990. Note that grade 8 daU reflects achievement prior to entering high school 
and as such constitutes an observation prior to any high school effect. This helps avoid the problem of 
confounding initial status and school effects discussed by Lee and Bryk (1989). 

Since the ability metric for these measures differ and because they were designed for different 
purposes, the decision was made to transform student scale scores to normal curve equivalents (NCEs). 
These are equal interval measures which will reflect a student's relative standing in terms of mathematics 
achievement for each of the years of the study. This scale is based on 1986 South Carolina norms. The 
number of students tested for the period of the study were roughly 46,000 in grade 8, (1987), 49,000 in 
gt-ade 9 (1988); 38,000 in grade 10 (1989); and 35,000 in grade 11 (1990). 

To satisy the data requirements of education programs mandated by the South Carolina legislature, 
the South Carolina Department of Education (SCDOE) maintains a data bank of matched case test 
records. Annually, student records are matched with test results of the previous year. For the current 
* study, these data greatly facilitated the task of following students throughout the period of interest. 
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Specifically, two data banks of matched student records were obtained. The first wss the grade 8, 1987 
and grade 9» 1988 matched data set, and the second was the grade 10, 1989 and grade 11, 1990 matched 
student data. These two files contained all the achievement data needed to follow the cohort over the four 
years of interest. To link the studekit records from these two data sets, a computer matching algorithm 
was used (Mandeville, 1990/91). This algorithm is used to produce the annual matched data banks for the 
South Carolina testing programs and has been found to yield satisfactory results. From the original 
matched daU sets for grades 8 and 9 (38,217 students) and grades 10 and 11 (30,699 students), the 
algoiithm matched 25961 students. This number was then reduced to 17951 students by eliminating those 
who were not in the same high school during grades 9-1 1. 
The Within-School Model 

The design of the present study requires that some index of the nk^gnitude of achievement differences 
in mathematics between white and black; high and low SES, and male and female students be obtained 
for each school. For this purpose we used the standardized mean difference (Glass, 1976), often called 
the "effect size" sutistic. This measure is defined as the difference b'^tween the mean mathematics 
achievement for the two groups contrasted (e.g. , white and black students), divided by their pooled 
standard deviation. It is typically interpreted as an index of the effect of group membership on the 
outcome of interest. In the present application,the following variance stablizing traasformation was 
applied to these measures (Hedges & Olkin, 1983). 



H = \n 



+8 



V8 



For each contrast the standardized mean difference was computed for each of the four years of 
achievement data examined in this study. For the white/blade contrast these indexes are referred to as 
RACE87, RACE88, RACE89, and RACE90. Mean differences were taken so that a positive value 
indicates that the math achievement for white students at a given school for the given year was higher 
than the comparable achievement of black students, a negative value indicates the opposite result and a 
value of zero indicates no difference. Similarly, for the SES contrast we have SES87, SES88, SES89, and 
SES90. Positive values of these variables indicate lower math achievement for low SES students, and for 
the female/male contrast (SEX87, SEX88, SEX89, and SEX90) positive values indicate lower 
achievement for female students. For each contrast only schools with at least 10 students in each group 
were retained for analysis. During the HLM analyses, it was determined that convergence was enhanced 
if the effects were multiplied by 100 so this was done. This does not change any of the substantive 
interpretations. 

To clarify the above discussion, the following example of how the effect size measures were actually 
operationalized is provided. The data are for a school where the RACE contast appears to have changed, 
at least to some extent, toward equity over the three year period. 







1987 


1988 


1 1989 


1990 


Black (N==:44) 


Mean NCE 


49.23 


52.56 


52.01 


52.18 


White (N=« 118) 


Mean NCE 


62.98 


64.20 


62.73 


62.57 


White-Black 


Mean Diff 


13.75 


11.64 


10.72 


10.39 




Pooled SD 


13.16 


14.53 


15.66 


15.10 


Glass 


Effect Size (G) 


1.04 


0.80 


0.68 


0.69 


Transformed 


Effect Size (H) 


0.36 


0.28 


0.24 


0.24 


Tiansf*100 


100*H 


36.00 


28.00 


24.00 


24.00 



Note that the Glass effect size G (and also the transformed version) tend to decrease over the 1988-90 
st\uiy period indicating the relative improvement of black students vs. their white counterparts. 
Obviously, however, the largest decrease took place during the first year that these students were in the 
school, a period during which both racial groups showed increasing means. Thereafter, changes were 
smaller and due to a slight decrease in the performance of the white group. The slope of the best fitting 
straight line to the H daU for this school is -.03 and the linear relationship accounts for about 82% of the 
variation, i.e., r^=.82 
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The Between*School Model 

The betweea-school characteristics considered in the present study fall into three categories: (1) 
school nomuitive climate, (2) school instructional resources or setting, and (3) school and community 
context. 

Normative Climate 

The c .^native climate or school effectiveness indicators used in the present study are from surveys 
conducted by the South Carolina Department of Education. The surveys were designed to assess parent, 
student, and teacher perceptions of a school's status on 6 indicators of school effectiveness adopted by 
the South Carolina Sute Board of Education. These are: 

1 . Instructional Leadership of the Principal 

2. School's Emphasis on Academics 

3. High Expectations of Student Achievement 

4. Positive School ClinuOe 

5. Frequent Monitoring of Student Progress 

6. Positive Home/School Relations 

Data for the present study are from the Spring 19S8 administration of these instruments at 
participating public schools across the state. In each setting a representative sample of students from 
grades 4 - 12, all teachers, and a san^le of parents were surveyed. These instruments consist of ten items 
for each of the 6 effectiveness indicators listed above. Data on the reliability and validity of the subscales 
of these instruments can be found in Segars and Gottesman (1989). Only student and teacher data were 
used in this study. 

Since the nuun focus of this investigation had to do with what might be called "equity* issues, rather 
than simply use the results for 12 student and teacher scales, individual survey items were inspected to 
identify those which might be especially appropriate to the egalitarian aspects of the study. Ten such 
items—three on the student survey and seven on the teacher survey-were identified. These items were: 

Student Survey 

21. Teachers and principals expect all students to learn as rauch as possible. (ExpecUtions) 

44. If a student fails a test, he has another chance to learn the material. (Monitoring) 

48. When students do not learn with one approach, the teacher uses another approach. (Monitoring) 

Teacher Survey 

23. Each student has an opportunity for success. (Expectations) 

24. Teachers feel accounUble for students who don't understand the work, (ExpecUtions) 

25. Teachers expect low achievers to respond as often as other students. (ExpecUtions) 
27. Unsuccessful students get extra help from teachers. (ExpecUtions) 

29. Low achievers receive as much praise as high achievers. (ExpecUtions) 

46. Instruction is altered as needed to accomodate the needs of individual studenU . (Monitoring) 

50. Low test scores result in curriculum changes to meed student needs. (Monitoring,) 

It was decided to consider wh^er these item subsets could be used to form subscales apart from the 
remainder of the items. Thus, the intercorrelations between the items in a potential scale and the scale 
aggregate (the mean was used) were inspected. Student items #44 and 48 were combined to form a 
variable called MONISS which correlated .92 and .87 (at the school level) with the two individual items. 
Similarly, the five items from the teacher survey which had to do with ExpecUtions were combined 
to form EXPTT and the item-total correlations ranged from .76 to .87. The two Monitoring items from 
the teacher survey were used to create MONITT which correlated .84 and .93 with the two component 
items. The lone item from the student survey having to do with ExpecUtions was renamed EXPSS. 

For each of these four proposed new variables, it is obvious that the reliability is less than for the 
scales based on the intended ten items. On the other hand, the fact that school means will be used in the 
analysis, ameliorates this problem to some extent and, given the exploratory nature of the study, it was 
decided to use the foivr variables identified above. Furthennorc, the original versions of the four scales 
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involved, i. e.. Expectations and Monitoving from each of the two surveys, were redefined to reflect 
only the "residue" items. They will be referred to as EXPES, EXPET, MONIS, and MONIT. With the 
possible exception of EXPET, which is based on only five items, these four scales should reasonably 
reflect their original meanings. The other eight scales were not changed and will be denoted as LEADT, 
ACADT, CLIMT, and HOMET for teachers, and LEADS, ACADS, CLIMS, and HOMES for student 
data. 

As far as availability is concerned, the effective schools data were not as readily available as would be 
desireable. Districts were given an option as to whether their data would be made available to 
researchers, and, unfortunately, some decided against it. A further problem arose because of a desire to 
insure that the data for a school be representative. That is» the researchers had no control over response 
rates so the approach taken was to determine whether reasonable percentages of the teachers and students 
had responded. While we did not have exact teacher or pupil counts, we used average daily attendance 
(ADA) which was available, and looked at the number of re^nd^ts in each category from each school 
in relation to the ADA. The criteria which were used were that responses be available for a minimum of 
10% of the students (in ADA) and that at least one teacher survey was available for each 25 students in 
ADA. Of the 110 high schools for which any survey data were available, nine were eliminated due to 
these restrictions. Thus, the maximum school sample size was reduced to 101. 

The school variables in the other two categories are briefly described below. The informaticn for each 
was obtained from data files maintained by the SCDOE. 

Instructional Setting 

Average Teacher Education (MEDUC). This measure indexes teachers formal education and has a range 

from 10 (high school GED) to 20 (Doctor of Philosophy). The values presented were averaged for all 

regular teachers at a given school for the years 1987-88 and 1988-89. 
Number of Math Courses (NCOURSE). This variable represents the total number of math courses 

offered at a school. This measure is considered an indicator of the capability of a school to meet the 

needs (both advanced and remedial) of students in the subject area of mathematics. These data 

represent offerings during the 1988-89 school-year, 
/ dvanced Math Course Enrollment (AM ATHE). This variable reflects the percent of students enrolled in 

advanced math courses relative to those enrolled in any math coui-ses during the 1988-89 school -year. 

Advanced math courses include Algebra I and II; Trigonometry; Calculus, etc. 
Female Advanced Math Course Enrollment (FAMATHE)* This variable parallels AMATHE but for 

females only. 

Student Retention Rate (RETENT). This variable indexes the percent of students repeating a grade. The 
percenUges averaged are grade 9, 1988, grade 10, 1989 and grade 11, 1990. 

School and Community Context 

Percent Black (PCTBLK). This measure was defined as the percent of students who recorded their race as 
"Black" during the annual spring testing. For this study the percentages represent data from spring 
1987. This year was chosen because it was felt that these data were the most accurate. 

Average Mathematics Achievement (MEANM ATH). This variable represents the average mathematics 
achievement at a given school. The averages are based on BSAP, CTBS and EXIT exam scores for 
the total matched san^>le (25999 students), converted to NCEs, and averaged over years and grades. 

Student Subility (STABLE). This variable is a measure of the degree of stability of a schools* student 
population. It was calculated by averaging the percent of students whose test records were matched in 
grade 9 and the percent whoae test records were matched in grade 10. 

Percent Free Lunch (PCTFRE). Tliis variable represents the percent of students in a school who reported 
participation in the free school lunch program. These data were averaged over grades 9, 10, and 11. 

School Size (SIZE). This variable represents the average daily membership for a given school averaged 
for grade 9, 1988, grade 10, 1989, and grade 11, 1990. 

Community Type (URBAN, SUBURB, RURAL). These three dummy variables index the type of 
community served by the school. The data represent principal reports of the origins (rural, urban, 
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mixed) of the majority of their student body. In Ublcs and analyses to follow, RURAL will not be 
included since it would be redundant. The current data are from the 1989 survey. 
Statistical Model 

In line with a previous study of elementary school pupils (Mandeville & Kennedy, 1991), the current 
effort employs an adapUtion of the growth curve formulation of Hierarchical Linear Modeling (HLM) as 
discussed by Bryk and Raudenbush (1987). Specifically, it is assumed that changes in the achievement 
discrepancy (blacks/whites females/males, low/high SES) within a school as students are promoted from 
grade 9 to grade 1 1, can be modeled as a function of a K-degree growth polynomial. In particular, if Yj^ 
is the achievement discrepancy in school i (i= l,.,.,n) at grade t, then 

where ny^ is the kth growth parameter for school i, ajj is year or grade, and the Rj^ are elements of a T 
dimensional random error vector assumed to be normally distributed with mean zero and a general 
covariance matrix Sj. The Y^j arc assumed to be in the same metric and normally distributed; the former 
assumption is met for these daU and the latter was tested using the Kolomogorov D statistic (Stephens, 
1974). 

It is expected that the rate of growth, and perhaps even the direction, will vary among schools. This 
variation is then modeled as a function of school level characteristics as follows: 

where ny^ is the kth parameter of the within-school nnxlel for school i, p = 1,,,., P-1 represents the 
number of bctween-school variables, depicts the relationship of the school level variable Xj^ to the 
kth within-school growth parameter, ana U^ is random error. The Uj^ are elements of a K-1 
dimensional vector which is presumed multivariate normal with mean vector 0 and dispersion matrix T. 
Thus, the ny^ are also presumed to follow the multi-normal distribution. This assumption was tested 
using a method described in Bryk, Raudenbush, Sdtzer, and Congdon (1989, p. 73). 

As noted by Bryk and Raudenbush (1987), this formulation has several attractive features: (1) an 
evaluation of the quality of a measure for assessing initial sUtus and change can be obtained, (2) an 
estimate of the correlation between initial status and rate of change corrected for measurement error is 
available, and (3) the proportion of parameter variability explained by school level predictors may be 
estimated. 

Analytic Strategy 

The first step in these analyses was to determine the degree of the polynomial for the within-school 
model. Based on a plot of the overall means for the entire sample, the determination can be made as to 
whether a linear or higher degree polynomial is appropriate. In the second step of the HLM analysis, an 
unconditional model (i.e., a model with no between school predictors) was fit to the dato. This step 
allowed for testing of the assumptions of normality and for determining if there was sufficient parameter 
variability in school growth coefficients, the ny^, to attempt to relate this variation to school level 
characteristics. Additionally, residuals produced in this step were used in an attempt to identify complex 
relationships such as interactions. In the final step of the analysis, bctwccn-school variables were 
included in the model as predictors of parameter estimates from the within-school model. Nonsignificant 
predictors were dropped from the between-school model which was then refitted as necessary until it 
contained only variables with significant (or, in a few casses, nearly significant) test statistics. 

Results 

Preliminary Results 

Table 1 presents demographic and achievement characteristics of the sample and the population of 
students tested in grades 8, 9, 10 and 1 1. The achievement daU are presented in the original scale score 



Table 1 about here 



metric. The figures in this table are informative of the degree to which the sample is similar to the larger 
population of students tested statewide. The demographic daU indicate that the sample has about the sarot 
makeup as the population as regards race and gender. The daU on die SES indicator (% Free Lunch) is 
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somewhat inconclusive but it spears that the two groups are comparable. Finally, while the sample 
appears to have outperformed the population as a whole in math during 1987 and 1989, the results for the 
other two years are much more comparable. In summary, the differences are not great and consistent 
with the rather restricted nature of the san:q)le. 

During the daU preparation, the aforementioned requirement that a minimum of ten student records 
for each level of each contrast be available in order for the effect size to be meaningful was applied. Of 
course, for some schools this requirement was not satisfied for one or more years so that that school 
could not be used for the analysis of the specific contrast under consideration. Since the schools so 
eliminated were not consist^t for the different contrasts, it would have been possible to use the different 
san^les of schools for the different contrasts. However, it was felt that it would be useful to obtain a 
single sample of schools for which a complete dato set were available for all three contrasts for all four 
years and for which all school-level predictors were available. The resulting sample was reduced to 79 
schools. Table 2 lists descriptive sUtistics for the school level predictors for these 79 schools. 



Table 2 about here 



Four of the variables listed represent three year averages, and two were computed by averaging two 
years of daU. The indicators with three years of daU were SIZE, PCTBLK, PCTFRE, and RETENT. 
The indicators with two years of daU were MEDUC and TURN. The use of averages implies that there 
have not been in^rtant changes in these variables over the period examined. To examine this 
assumption, standard deviations were computed for each of the six variables for each school and plots 
were constiaicted. These plots were positively skewed with a preponderance of small values in the lower 
tail. These results indicate that for the majority of schools, year-to-year variations in the six indicators 
were slight. 

Pearson correlations among school level variables were generally consistent with previous reports 
and, therefore, are not presented. For example, correlations among the normative climate measures were 
generally high for both the teacher and student scales, with values above .40 typical (for similar results 
see Kijai, 1990). On the other hand, correlations among the instructional and context variables were often 
in the moderate range and tended to rettect typical patterns (e.g. , the percenUge of free lunch students 
had a statistically significant relationship with mean achievement level). 

Table 3 liste descriptive statistics for the standardized mean difference (100*H) for the three contrasts 
studied for the years 1987-1990, While there is some fluctuation, the mean differences for RACE and 

Table 3 about here 



SES tend to be remarkabl ' suble over the three high school years (1988-1990). Thus, the hoped for 
decreasing trend did not occur. In each instance, the effects are positive, indicating a tendency for low 
SES and black students to have lower levels of mathematics achievement than their upper SES and white 
counterparts.For refeitaice, in the original (G) metric, values of 100*H of 20 and 25 are equivalent to 
effects of about .6SD and .7SD respectively. The results for SEX yield almost no effect for each of the 
years considered. While this is unexpected, there is some evidence that sex differences in mathematics 
have began to disappear in recent years (Willms & Kerr, 1987). Finally, while the averages for the 
various contrasU do not show much year-to-year change, it is important to note that in each case there is 
considerable variation among schools in the magnitude of a given effect as indicated by the standard 
deviation and range sUtistics. 

Table 4 presents ranges of correlations of school level predictors with standardized mean differences 
for each contrast for each of the high school years, i.e., 1988-1990. The rcsulte are cncouragmg for 
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proponents of the type of school effectiveness indicators utilized in the current study (Edmonds, 1981). 
Several scales from the student questionnaire (but only one from the teacher survey) are consistent 
predictors of achievement differences between black and white students, and students from low and upper 
socioH5conomic backgrounds. For the SES contrasts the magnitude of these correlations nval those of the 
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mean achievement level at the school and the percent of low SES students. Additionally, while school 
mean achievement level (MEANMATH) and enrollment in advanced math courses (AMATHE) yield 
positive correlations with the SES contrasts-indicating that as average achievement and advanced math 
enrollment in a school increases the relative standing of low SES students deteriorates--the student 
climate scales yield negative measures indicating a more equitable outcome. These statements also apply, 
but to a lesser extent^ to the RACE contrast. These results are supportive of school effectiveness 
measures as indexes which can differentiate between schools with respect to the social distribution of 
student achievement. However, the data presented are at most correlational and can not be taken as 
evidence that these measures actually induce the more equitable distribution. This particular problem, 
attributing causality, has plagued much of the school effects literature. For example, in a recent paper 
Lee and Bryk (1989), despite the sophisticated multilevel techniques employed, urge caution in 
attributing their results to ^'school effects". The position adopted in this study is that inference regarding 
causality requires that changes in the social distribution of achievement be linked to school 
characteristics. The HLM results presented below are directed toward this problem. 
HLM Results 

One of the first considerations in the growth application of HLM involves determining the degree 
(linear, quadratic^ etc.) of the polynomial for the within-school model. For the current application, the 
standard practice of plotiiug the sample averages for the within-unit data was followed. These plots 
indicated that a linear model was adequate for these data. To further facilitate interpretation of results, 
the within-school time variable was coded to reflect years beyond the high school entry point of 1987. 
With this coding, the intercept of the within-school model reflects a particular contrast prior to entering 
high school, 1987, and the slope or linear term represents rate of change over grades 9, 10, and 1 1. 

Part one of the analysis involv;xi fitting the so-called "unconditional" model to the daU. These results 
were used to assess the tenability of the normality as5Uiiq>tion regarding the within-school estimates. For 
each of the three contrasts, no si^ficant departures from normality were noted. The unconditional 
model also yielded tests of the hypotheses that the intercept and linear term of the within model were not 
significantly different from zero. As presented in Table 5, while the intercept terms are all clearly 
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significantly different from zero, the results for the linear term involviiig the SEX contrast does not 
achieve significance. This result is not problematic, however, since the primary focus is on explaining 
variation among schools around the average growth rate or linear term. The results for the intercepts 
indicate that, at the end of grade 8, white, high SES, and to a much lesser extent, males, were achieving 
at higher levels than black, low SES, and female students. The significant mean slope estimates are also 
positive for RACE and SES, indicating a worsening of these conditions over time. However, when these 
estimates of .72 and .61 units per year are transformed back to the original Glass effect size metric, they 
correspond to the rather modest value of .02SD. 

Table 6 presents estimates of parameter variance for the various contrasts studied. Because only 
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true parameter variance is potentially explainable, the results of this table are used to determine if an 
attempt should be made to model variation in specific parameters. The Chi-square sUtistics indicate that 
there is significant parameter variability associated with the intercept and slope for the RACE and SES 
contrasts but that only the intercept for SEX has significant parameter variability. The "reliability" daU 
presented in the Uble indicate the proportion of total variability (parameter variability + san^ling error) 
for terms of the within-school model which is paranster variability. Large values mean that most of the 
observed differences between schools reflects true d«stmctions, and small values liuggest that most school- 
to-school differences are due to sampling error. The reliabilities for the intercepts are typically large, but 
for the slope values, especially for the male/female contrast, they tend to be small. This latter result will 
lessen the likelihood of finding significant between-school predictors. In fact, the small reliability for the 
SEX slope coupled with the fact that the parameter variance is nonsignificant, indicates that the linear 
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term should be considered "fixed" and, therefore, not subject to future modeling. Since the slopes are 
the parameters of primary interest in this study, the SEX contrast will not be considered further. The 
final statistics presented in Table 6 are estimates of the correlation of the intercept (initial sUtus) and 
linear (rate of growth) terms corrected for error. These values are negative for RACE and SES and not 
meaningful for SEX. The implication for the SES contrast is that in schools where the prc-high school 
achievment for low SES students was relatively low, the tendency is toward a more equiuble distribution 
as the cohort moved from grade 9 to grade 11 (r = -,51). A similar but weaker (r « -.24) conclusion is 
implied for black students. 

Fitted HLM models, following the procedures described above, are presented for the RACE and SES 
contrasts in Tables 7 and 8. Each of the two contrasts had some school level predictors of growth rate. 

Tables 7 and 8 about here 



For the white/black slope, three predictors--SUBURB, STABLE, and EXPSS-had significant negative 
coefficients. Apparently MONISS. while nonsignificant, acts as a supressor for EXPSS since if MONISS 
is excluded from the model, EXPSS is no longer significant. Thus, the interpretation of these effective 
schools variables is a bit ambiguous. Negative signs on these coefficients indicate that larger values of the 
predictors were associated with reductions in the values of the contrasts. Thus, a significant reduction in 
the RACE contrast is to be found in suburban schools and schools with more sUble student populations. 
If EXPSS is to be interpreted, this would mean that schools in which students perceive that principals and 
teachers have high expectations for all studente are also schools where white/black differences are 
decreasing. At the bottom of Table 7. it is reported that these predictors account for biightly over half 
(50.29%) of the true variance in the slope parameter, a surprisingly large value. 

Turning to the slope for the SES contrast, the significant predictors again include SUBURB, (only 
significant at the p=. 10 level, however) but this time along with the percent of students eligible for free 
lunch. Again, the signs are negative leading to an interpretation that schools with larger percentages of 
low SES children do tend to address the needs of these low SES students. This is not surprising, since it 
is possible that these low SES students make up a majority of the school. It might be conjectured that, in 
some cases, the needs of the (possibly small group oO high SES students are not well met so that a 
lowering of the value of the contrast is due to increased achievement for the low SES students but little or 
no progress for the high SES group. These two predictors account for a little over one-fourth (26.51 %) 
of the true variance in the SES slope parameter. 

The fitted models for the intercepts are of less interest in the present formulation, for they represent 
contrasts prior to entering high school. The fact that an effective schools variable is predictive of the 
intercept in both Tables 7 and 8 is curious. This says that high schools with relatively high values on 
these two ES variables begin with RACE (Table 7) and SES (Table 8) groups which are not as discrepant 
in terms of math achievement as the typical high school. One possibility is that some district level factor 
is operating. The significance of RETENT for the RACE contrast might be a spurious reflection of the 
research design used here. i.e. . the fact that only data for continuously promoted students were used. 

Summary and Conclusions 

Unlike many previous studies in the school effects or effectiveness literature, this study was 
concerned with finding school level correlates of changes in the social distribution of achievement 
defined as differcncca in mathematics achievement for black and white students, female and male 
studente, and students from lower and upper socioeconomic backgrounds. The list of school 
characteristics considered was extensive and covered three general areas: school /community context, 
school normative climate, and school instructional setting. Most of the variables considered are similar 
to those frequently used in this literature and many of the results of this study which examined school-to- 
school variations at one point in time supported previous conclusions. In particular, studwit normative 
climate scales were found to correlate well with achievement differences between students from low and 
upper socioeconomic backgrounds and, to a more limited extent, with achievement differences between 
black and white studente. However, when the emphasis changed to "rate of change" in the three 
contraste considered, only expectations as expressed by studente (the specialiied variable EXPSS) was 
found to be a significant predictor of changes in the slope for the RACE contrast, and this result is not 
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even veiy clear-cut. While selected ES variables were also demonstrated to be related to intercepts for 
both RACE and SES, the meaning of these results is unclear (because the intercepts reflect pre-high 
school conditions) and are at best only consistent with a 'point in time" rather than a "trend" 
interpretation. 

The results provide some evidence that suburban schools have been better able to promote equity 
(both vis*s*vis RACE and SES) than schools located in a rural or urban setting. Further, the stability of 
the student population and the proportion of disadvantaged students in the school also had a significant 
effect on the social distribution of math achievement. 

At the begining of this paper it was noted that many school improvement programs are based on 
indicators generated from the school effects and school effectiveness literature. Use of these measures is 
generally based on the assun^tion that if schools improve on these indicators, achievement and the 
distribution of achievement will improve. This study, in addition to exploring other characteristics of 
schools, attempts to test this assumed causal link between effectiveness indicators and changes in the 
distribution of student achievement. The results are not generally supportive of a causal link and may 
help explain the of^n disappointing results associated with these indicators (Miller, Cohen Sc Sayre, 
1985). However, schools are especially complex organizations. The current formulation, while not in 
conflict with, does not explore interesting questions about student characteristics which can be explored 
with newer three-level multilevel models (see Bryk & Raudenbush, 1988). Additionally, the school 
indicators studied were static in that they reflected averages and not changes which might have occurred 
over the years of the study. The assumption implied by this operational definition may not be tenable. 

The (possible) significance of the variable EXPSS, tailored as it was to focus on equity, suggests that 
general ES measures are not sufficient to deal with the issues of this study. Finally, the fact that data 
derived from teachers provided few significant correlations with the contrasts even at a point in time 
(Table 4), suggests that students are the naore relevant source for this information. 
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Table 1 

Demographic and Achievement Characteristics for Population and Matched Sanq^le. 



1 Variable 




Population 




Matched Sample 


—I 


lYear | 


1987 


1988 


19891 


1990 


1987 1988 


1989 


1990 


Ciiaracteristics 


% Black 


39 


40 


38 


37 


40 40 


40 


40 


% Free Lunch * 


31 


NA 


19 


NA 


24 24 


24 


24 


%Male 


51 


51 


49 


49 


48 48 


48 


48 


Mean Math 


748 


732 


778 


741 


774 738 


791 


742 



Note: The number of students in the population was 45»636 in 1987» 48»901 in 1988» 38>246 
in 1989, and 24,891 in 1990. The number of students in the matched sample was 17,951. 
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Table 2. 

Descriptive Statistics for School Level Predictors (n=79) 





M 


SD 


Min 


Max 


i>iornuiuTe 
































LEADT 


4.U.) 




0.26 




3.0 




4.6 


ACAD I 






0.15 




3.5 




4.4 


bXrbi 






0.14 




3.7 




4.3 


EXFTl 


fin 




0.16 




3.5 




4.2 


CLIMT 






0.32 




2.7 




4.4 


MONrr 


3.73 




0.19 




3.4 




4.2 




3.80 




0.19 




3.4 




4.3 


rlOMbl 


3.95 




0.22 




3.4 




4.4 


jHionnaiiTe 
















CI iinaic*diuoen i 
















T "c A r>o 
LbAU^ 


3.47 




0.22 




2.9 




4.2 




3.34 




0.13 




3.1 




3.7 


bXPbS 


3.29 




0.14 




3.0 




i.o 


EXPSS 


4.07 




0.20 




3.5 




4.4 


CLIMS 


3.04 




0.27 




2.5 






MONIS 


3.44 




0.15 




3.1 




3.7 


MONSS 


2.87 




0.20 




2.5 




•J 1 
J.J 


HOMES 


3.34 


0.17 




3.0 




Instnictinal 
















Settlns 
















MEDUC 


17.18 ' 


0.24 


16.4 


17 


RETENT 


8.19 


6.32 


1.0 


43.0 


NCOURSE 


12.46 


1.98 


9.0 


18.0 


AMATHE 


45.56 


10.90 


23.0 


84.0 


FAMATHE 


48.34 


10.79 


23.0 


86.0 


School/Community 














Context 
















MATHMEAN 


57.62 


4.38 


47.6 


67.2 


SIZE 


249.52 


123.33 


61.0 


634.0 


PCTBLK 


41.50 


18.18 


8.0 


89.0 


PCTFRE 


20.90 


13.86 


3.0 


61.0 


STABLE 


89.40 


4.34 


69.0 


97.0 


URBAN 


0.29 


0.46 


0.0 


1.0 


SUBURB 


0.34 


0.48 


0.0 


1.0 
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Table 3. 

Descriptive Statistics for Standardized Mean Differences (lOO^H) 
By Contrast and Year (n-79 schools) 



i,rOni-rasi* X car 


ivi 


! 


^/f in 


IVldX 


White/Black-1987 


23.92 


8.29 


2.35 


46.44 


VUlkSfA/nickAb- IQBfi 

wiute/uiaCK-iyoo 


ZD. 35 








White/Black-1989 


27.70 


6.79 


13.48 


48.60 


White/Black-1990 


25.95 


8.90 


1.13 


52.80 


High/Low SES-1987 


20.16 


10.15 


-.93 


45.95 


High/Low SES-1988 


22.22 


10.67 


-2.68 


47.02 


High/Low S£S*1989 


21.58 


9.37 


1.10 


40.92 


HighAx>w S£S'1990 


22.41 


9.36 


-1.98 


47.17 


MaIe/Fmale-1987 


3.57 


7.23 


-24.44 


16.72 


Male/Fmale-1988 


.23 


6.59 


-20.49 


14.58 


Male/Feinale-1989 


5.78 


7.23 


-15.09 


20.58 


Male/FemaIe-1990 


.65 


6.76 


-17.93 


15.49 
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Table 4. 

Ranges of Pearson Conelations Between Each of the Three Standardized Differences 
and School Level Predictors for 1988, 1989 and 1990. 



Variable 


RACE88-90 I 


SES88-90 


SEX88-90 


Normative 








Clunate-Teacher 








LEADT 


-.08 to .03 


-.11 to -.02 


-.11 to .00 


ACADT 


-.03 to .03 


-.09 to .06 


-.10 to ,01 


EXPET 


.09 to .11 


.03 to .12 


-.16 to -.01 


EXPTT 


.04 to .11 


.02 to .08 


-.07 to .08 


CLIMT 


-.07 to .01 


-.05 to -.02 


-.13 to -.06 


MONIT 


-.07 to -.01 


-.06 to .02 


-.13 to .02 


MONTT 


-.09 to -.04 


-. 13 to -.04 


-.09 to .05 


HOMET 


.03 to. 11 


.12 to .22* 


-.14 to -.08 


Normative 








Climate-Student 








LEADS 


-.25 to -.11* 


-.41 to -.36** 


-.27 to -.17* 


ACADS 


-.09 to .01 


-.17 to -.15 


-.05 to -.01 


EXPES 


-.11 to -.03 


-.22 to -.11* 


-.10 to -.07 


EXPSS 


-.38to-.16** 


-.45 to -.32** 


-.23 to-.ll* 


CLIMS 


-.10 to .00 


-.08 to -.07 


-.02 to .01 


MONIS 


-.19 to -.13 


-.33 to -.29** 


-.10 to -.06 


MONSS 


-.19 to -.07 


-.40 to -.35** 


-.21 to -.16 


HOMES 


-.19 to -.06 


-.34 to -.29** 


-.24 to -.19* 


Instnictinal Setting 


MEDUC 


.01 to. 10 


.09 to. 17 


-.21 to -.09 


RETENT 


-.15 to -.09 


-.19 to .08 


-.10 to .01 


NCOURSE 


-.10 to -.03 


.20 to .27* 


-.04 to -.04 


AMATHE 


.18 to .27* 


.33 to .48** 


-.11 to .05 


FAMATHE 


.17 to .26* 


.30 to .44** 


-.18 to -.03 


School/Community 








Context 








MATHMEAN 


.13to.34** 


.34 to .49** 


-.02 to .07 


SIZE 


-.01 to .01 


.13 to .22* 


-.11 to -.10 


PCTBLK 


-.18 to -.07 


-.33 to -.29** 


-.01 to. 12 


PCTFRE 


-.19 to-.ll 


-.35 to -.21** 


-.05 to .14 


TURN 


-.13 to .06 


-.16 to -.03 


.11 to. 14 


URBAN 


.14 to .22* 


.12 to .30** 


-.03 to .02 


SUBURB 


-.17 to -.13 


-.01 to .08 


-.17 to -.03 



Note: * p < .05, ** p < .01 (both two-tailed) for at least one of the three correlations. 
The critical values of r which were used were .220 and .286 for the .05 and .01 levels* 
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Table 5. 

Estimated Mean Growth Parameters. 



Contrast 


1 Parameter 


Estimate 


SE 


T 


White/Black 












Intercept 


24.9571 


.8708 


28.66** 




Linear 


.7221 


.2680 


2.70* 


High/LowSES 












Intercept 


20.6718 


1.1658 


17.732** 




Linear 


.6118 


.2816 


2.172* 


Male/Female 












Intercept 


2.8583 


.7330 


3.90** 




Linear 


-.2767 


.2349 


-1.18 



Note: *p<.05. **p<.01 



Table 6 

Estimated Parameter Variability, Reliabilities and 
True Correlations of Initial Status and Growth. 



Contrast 

Parameter 

(Reliability) 


Estimates of Degrees of Chi 
Parameter Freedom Square 
Variance 


White/Black 

Intercept 
(Reliability) 

Lin.\: r 
(Reliability; ' 
Estimated True r 
Status & Growth 


45.07 78 315.1** 
(.752) 

1.44 78 104.4* 

(.253) 
-.24 


High/Low SES 

Intercept 
(Reliability) 
Linear 
(Reliability) 
Estimated True r 
SUtus & Growth 


94.26 78 639.0** 
(.878) 

2.52 78 130.5** 
(.253) 
-.51 


Male/Fenude 

IntercqH 
(Reliability) 
Linear 
(Reliability) 
Estimated True r 
Sutus 8c Growth 


27.79 78 236.22** 
(.654) 

.15 78 48.8 
(.034) 
NA 



Note: *p<.05, **p<.01 



?1 
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Table 7 

Fitted HLM Model for White/Black Differences 
in Mathematics Achievement. 



Parameter 


Estimate 


SE 


T 


Intercept 

RETENT 
EXPSS 


-.248 
-8.734 


.119 
4.236 


-2.08* 
-2.06* 


Variance Explained 
Percent 
of Total 
Percent 
Parameter 


5.81 
7.72 






Linear 

SUBURB 
STABLE 
EXPSS 
MONISS 


-1.157 
-.131 

-3.118 
2.030 


.464 

.051 
1.536 
1.472 


-2.49* 
-2.56* 
-2.03* 
1.38 


Variance Explained 
Percent 
of Total 
1 Percent 
1 Parameter 


12.73 
50.29 







Note: ! p<.10,* p<.05, **p<.01 



Table 8 

Fitted HLM Model for High/Low SES Differences 
in Mathematics Achievement. 



Parameter 


Estimate 


SE 


T 


Intercept 
MEANMATH 
LEADS 


.577 
-10.446 


.225 
4.162 


2.57* 
-2.51* 


Variance 

Percent 
of Total 
Percent 
Parameter 


Explained 
10.67 

12.15 


1 




Linear 

PCTFRE 
SUBURB 


-.053 
-.863 


.016 
.444 


3.27** 
1.94! 


Variance 

Percent 
of Total 
Percent 
Parameter 


Explained 
10.67 

26.51 







Note; !p<.10,*p<.05, **p<.01 



?2 



